Abstract

Medicine's current evidence base is insufficient for many of the decisions made daily by clinicians, patients, purchasers, and policy makers. The Patient-Centered Outcomes Research Institute represents an effort by the US government to address this shortcoming by funding comparative effectiveness research. Prioritizing that research will be a critical first step. To better understand components of an optimal process, we reviewed the literature and conducted interviews regarding the prioritization efforts of leading public and private health organizations in the United States and abroad. From this review, we propose a framework for prioritization, pose and answer key questions, and make recommendations regarding application of that framework. We also recommend that during the priority-setting process, there should be transparent conversations among those who make decisions about the priorities and the public.
Hide

Abstract

Comparative effectiveness research (CER) is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat and monitor a clinical condition, or to improve the delivery of care. The purpose of this article is to compare - within the scope of CER - the value of implementation and drug trials. Implementation trials have limitations similar to drug trials in terms of generalizability of results outside the trial setting and ability to identify best practice. However, in contrast to drug trials, implementation trials do not provide value in terms of ruling out harm, as implementation strategies are unlikely to cause harm in the first place. Still, implementation trials may provide good value when there is a high error probability in deciding whether implementation will be cost effective or if costs associated with making an erroneous decision are high. Yet the low risk of implementation programmes to cause harm may also allow for alternative approaches to identify best implementation practice, perhaps outside the scope of rigorous trials and testing. One such approach that requires further investigation is a competitive market for quality of care, where implementation programmes may be introduced without prior evaluation.
Hide

Abstract

Comparative effectiveness research (CER) is increasingly popular, yet discussions of its conduct and consequences often overlook the extensive history of comparing different therapeutic options in patient-oriented research. In particular, research in the Department of Veterans Affairs (VA) has included a decades-long focus on generating information that can enhance medical decision making and improve health outcomes. Categories of such research include multisite randomized controlled trials (conducted by the Cooperative Studies Program) and observational studies involving either primary or secondary data collection. As representative examples from cardiology, a landmark VA clinical trial published in the 1970s evaluated the benefits of coronary artery bypass grafting surgery among patients with angina; a VA trial initiated in the 1990s, and identified formally as CER, demonstrated that percutaneous coronary intervention is not superior to optimal medical therapy; and a database investigation using information from the VA electronic medical record system in the 2000s found that use of proton pump inhibitor medication is associated with the attenuation of the benefits of clopidogrel among patients hospitalized for acute coronary syndrome. A review of these (and other) selected projects, based on their type of study design, serves to highlight the strengths, limitations, and potential of CER.
Hide

Abstract

Comparative effectiveness research (CER) can provide valuable information for patients, providers and payers. These stakeholders differ in their incentives to invest in CER. To maximize benefits from public investments in CER, it is important to understand the value of CER from the perspectives of these stakeholders and how that affects their incentives to invest in CER. This article provides a conceptual framework for valuing CER, and illustrates the potential benefits of such studies from a number of perspectives using several case studies. We examine cases in which CER provides value by identifying when one treatment is consistently better than others, when different treatments are preferred for different subgroups, and when differences are small enough that decisions can be made based on price. We illustrate these findings using value-of-information techniques to assess the value of research, and by examining changes in pharmaceutical prices following publication of a comparative effectiveness study. Our results suggest that CER may have high societal value but limited private return to providers or payers. This suggests the importance of public efforts to promote the production of CER. We also conclude that value-of-information tools may help inform policy decisions about how much public funds to invest in CER and how to prioritize the use of available public funds for CER, in particular targeting public CER spending to areas where private incentives are low relative to social benefits.
Hide

Abstract

BACKGROUND: In the debate on improving the quality and efficiency of the United States healthcare system, comparative effectiveness research is increasingly seen as a tool for reducing costs without compromising outcomes. Furthermore, the recent American Recovery and Reinvestment Act explicitly describes a prioritization function for establishing a comparative effectiveness research agenda. However, how such a function, in terms of methods and process, would go about identifying the most important priorities warranting further research has received little attention. OBJECTIVES: This study describes an Agency for Healthcare Research and Quality-funded pilot project to translate one current comparative effectiveness review into a prioritized list of evidence gaps and research questions reflecting the views of the healthcare decision makers involved in the pilot. METHODS: To create a prioritized research agenda, we developed an interactive nominal group process that relied on a multistakeholder workgroup scoring a list of research questions on the management of coronary artery disease. RESULTS: According to the group, the areas of greatest uncertainty regarding the management of coronary artery disease are the comparative effectiveness of medical therapy versus percutaneous coronary interventions versus coronary artery bypass grafting for different patient subgroups; the impact of diagnostic testing; and the most effective method of developing performance measures for providers. CONCLUSIONS: By applying our nominal group process, we were able to create a list of research priorities for healthcare decision makers. Future research should focus on refining this process because determining research priorities is essential to the success of developing an infrastructure for comparative effectiveness research.
Hide

Abstract

Background- Comparative effectiveness reviews offer a systematic method to critically appraise existing research and to identify unaddressed clinical areas in cardiovascular disease where significant morbidity, mortality, and variation in the use of resources persist. To delineate and help select areas where comparative effectiveness reviews are needed, the Effective Health Care Program of the Agency for Healthcare Research and Quality involved stakeholders in prioritization of the research agenda. Methods and Results- We involved a diverse panel of stakeholders representing a broad range of clinical, policy, and patient perspectives. To assist in prioritization of topics for evidence synthesis, we created a framework evaluating 12 cardiovascular disease subcategories that reflect American College of Cardiology/American Heart Association disease-based guidelines. We performed an environmental scan for each disease subcategory to populate this framework with existing knowledge, levels of evidence, and degrees of public interest. Through a formalized process, 4 disease subcategories were prioritized: chronic coronary artery disease, ventricular arrhythmias, heart failure, and cerebrovascular disease. Within these subcategories, 11 topics that address the comparative safety and effectiveness of existing treatments and evaluate emerging treatments were nominated by the stakeholder panel to proceed for feasibility assessment before developing comparative effectiveness reviews. Conclusions- Using a systematic process deriving consensus from multiple stakeholders across cardiovascular disease states, we generated a prioritized list of evidence synthesis topics to inform decision makers. The topics vetted through this process seek to determine the comparative safety and effectiveness of a range of treatments, both established and emerging, and are immediately relevant for prevalent disease states.
Hide

Abstract

PURPOSE:: To set priorities for new systematic reviews (SRs) and randomized clinical trials on the management of primary angle closure (PAC) using clinical practice guidelines and a survey of Asia-Pacific clinicians. METHODS:: We restated the American Academy of Ophthalmology's Preferred Practice Patterns recommendations for management of PAC into answerable clinical questions. We asked participants at the Asia-Pacific Joint Glaucoma Congress 2010 in Taipei to rate the importance of having an answer to each question for providing effective patient care, using a Likert-type scale and scoring from 0 (not important at all) to 10 (highly important). We identified relevant SRs and mapped the evidence to clinical questions to identify evidence gaps. RESULTS:: We generated 42 clinical questions. One hundred seventy-five individuals agreed to participate in the survey, 132 responded (75.4% response rate) and 96 completed the questionnaire (54.9% usable response rate). Questions rated important include laser iridotomy for the prevention of angle closure in primary angle-closure suspects, further therapies in eyes with plateau iris syndrome after laser iridotomy, and evaluation of the fellow eye in acute angle-closure patients for improving prognosis. Up-to-date and conclusive SR evidence was not available for any of the 42 clinical questions. CONCLUSIONS:: We identified high priority clinical questions on the management of PAC, none of which had reliable SR evidence available. New SRs and randomized clinical trials can be initiated to address these evidence gaps.
Hide

Abstract

BACKGROUND: Comparative effectiveness research (CER) is an emerging field that compares the relative effectiveness of alternative strategies to prevent, diagnose, or treat patients who are typical of day-to-day practice. We developed a priority list of CER topics for inflammatory bowel disease (IBD). METHODS: Following the Institute of Medicine's approach, we developed and administered a survey to gastroenterologists asking for important CER topics in IBD. Two patient focus groups were convened to solicit additional CER studies. CER topics were presented to the expert panel using the RAND/UCLA methodology. Following initial ratings, the panel met to discuss and re-rate priorities. The top 10 CER topics were identified using a point-allocation system. RESULTS: Responses were collated into 234 CER topics across 21 categories, of which 87 were prioritized for discussion and re-rated. Disagreement regarding priorities was observed in 5 of 87 studies. We utilized a point-allocation system to prioritize the top-10 CER topics. These related to comparing the effectiveness of: biomarkers in IBD; withdrawal of anti-tumor necrosis factor (TNF) or immunomodulators for Crohn's disease in remission; mucosal healing as an endpoint of treatment; infliximab levels versus standard infliximab dosing; anti-TNF monotherapy versus combination therapy in patients failing thiopurines; safety of long-term treatment options; anti-TNF versus thiopurines for prevention of postoperative recurrence; and treatment options for steroid-refractory UC. CONCLUSIONS: We systematically developed a list of high-priority IBD topics for CER based on a survey of gastroenterologists, expert review, and patient input. This list may guide IBD research toward the most important CER studies.
Hide

Abstract

ABSTRACT: Systematic reviews comparing the effectiveness of strategies to prevent, detect, and treat chronic kidney disease are needed to inform patient care. We engaged stakeholders in the chronic kidney disease community to prioritize topics for future comparative effectiveness research systematic reviews. We developed a preliminary list of suggested topics and stakeholders refined and ranked topics based on their importance. Among 46 topics identified, stakeholders nominated 18 as 'high'priority. Most pertained to strategies to slow disease progression, including: (a) treat proteinuria, (b) improve access to care, (c) treat hypertension, (d) use health information technology, and (e) implement dietary strategies. Most (15 of 18) topics had been previously studied with two or more randomized controlled trials, indicating feasibility of rigorous systematic reviews. Chronic kidney disease topics rated by stakeholders as 'high priority'are varied in scope and may lead to quality systematic reviews impacting practice and policy.
Hide

Abstract

OBJECTIVE: To use information about prevalence, cost, and variation in resource utilization to prioritize comparative effectiveness research topics in hospital pediatrics. DESIGN: Retrospective analysis of administrative and billing data for hospital encounters. SETTING: Thirty-eight freestanding US children's hospitals from January 1, 2004, through December 31, 2009. PARTICIPANTS: Children hospitalized with conditions that accounted for either 80% of all encounters or 80% of all charges. MAIN OUTCOME MEASURES: Condition-specific prevalence, total standardized cost, and interhospital variation in mean standardized cost per encounter, measured in 2 ways: (1) intraclass correlation coefficient, which represents the fraction of total variation in standardized costs per encounter due to variation between hospitals; and (2) number of outlier hospitals, defined as having more than 30% of encounters with standardized costs in either the lowest or highest quintile across all encounters. RESULTS: Among 495 conditions accounting for 80% of all charges, the 10 most expensive conditions accounted for 36% of all standardized costs. Among the 50 most prevalent and 50 most costly conditions (77 in total), 26 had intraclass correlation coefficients higher than 0.10 and 5 had intraclass correlation coefficients higher than 0.30. For 10 conditions, more than half of the hospitals met outlier hospital criteria. Surgical procedures for hypertrophy of tonsils and adenoids, otitis media, and acute appendicitis without peritonitis were high cost, were high prevalence, and displayed significant variation in interhospital cost per encounter. CONCLUSIONS: Detailed administrative and billing data can be used to standardize hospital costs and identify high-priority conditions for comparative effectiveness research--those that are high cost, are high prevalence, and demonstrate high variation in resource utilization.
Hide

Abstract

BACKGROUND: Systematic approaches to stakeholder-informed research prioritization are a central focus of comparative effectiveness research. Genomic testing in cancer is an ideal area to refine such approaches given rapid innovation and potentially significant impacts on patient outcomes. OBJECTIVE: To develop and pilot test a stakeholder-informed approach to prioritizing genomic tests for future study in collaboration with the cancer clinical trials consortium SWOG. METHODS: We conducted a landscape analysis to identify genomic tests in oncology using a systematic search of published and unpublished studies, and expert consultation. Clinically valid tests suitable for evaluation in a comparative study were presented to an external stakeholder group. Domains to guide the prioritization process were identified with stakeholder input, and stakeholders ranked tests using multiple voting rounds. RESULTS: A stakeholder group was created including representatives from patient-advocacy groups, payers, test developers, regulators, policy makers, and community-based oncologists. We identified 9 domains for research prioritization with stakeholder feedback: population impact; current standard of care, strength of association; potential clinical benefits, potential clinical harms, economic impacts, evidence of need, trial feasibility, and market factors. The landscape analysis identified 635 studies; of 9 tests deemed to have sufficient clinical validity, 6 were presented to stakeholders. Two tests in lung cancer (ERCC1 and EGFR) and 1 test in breast cancer (CEA/CA15-3/CA27.29) were identified as top research priorities. CONCLUSIONS: Use of a diverse stakeholder group to inform research prioritization is feasible in a pragmatic and timely manner. Additional research is needed to optimize search strategies, stakeholder group composition, and integration with existing prioritization mechanisms.
Hide

Abstract

FROM THE INTRODUCTION: ".... The Agency for Healthcare Research and Quality (AHRQ) is the lead Federal agency charged with improving the quality, safety, efficiency, and effectiveness of health care for all Americans. As 1 of 12 agencies within the Department of Health and Human Services, AHRQ supports health services research to improve the quality of health care and promote evidence-based decisionmaking; AHRQ supports a variety of CER/PCOR activities through its Effective Health Care (EHC) Program.2 The EHC Program funds individual researchers, research centers, and academic organizations that work with AHRQ to produce effectiveness and comparative effectiveness research for clinicians, consumers, and policymakers. To contribute to its agenda for CER/PCOR activities related to mental health, AHRQ contracted the RTI International-University of North Carolina (RTI-UNC) Evidence-based Practice Center (EPC) and the Scientific Resource Center Stakeholder Engagement Team to engage a broad and representative group of stakeholders to discuss issues related to serious mental illness (SMI) in a series of three meetings (Issues Exploration Forum [IEF]), as detailed in the methods section. The results of three large comparative effectiveness trials have been sobering, and arguably have highlighted the limitations of our current ability to help many patients afflicted with schizophrenia, bipolar disorder, and depression. The Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) study reported that an older typical antipsychotic medication was as effective as the newer, more publicized atypical antipsychotics when used as a first-line treatment. In addition, nonadherence to medication was the norm, not the exception, with almost 75 percent of patients not using their medication after 18 months of treatment.3 The Systematic Treatment Enhancement Program for Bipolar Disorder (STEP-BD) trial reported that about 40 percent of bipolar patients do not recover from a manic episode.4 In addition, for those who do recover, the relapse rates for either a subsequent depressive or manic episode were about 50 percent over the following 2 years. In the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial, after 13 weeks of treatment with a selective serotonin reuptake inhibitor only a third of patients met remission criteria.5 Furthermore, after another 39 weeks of treatment only another third of the initial cohort remitted, leaving a full third of patients symptomatic.6 Reports from the Schizophrenia Patient Outcomes Research Team (PORT) underscore the point that effective treatments are available, but implementation and uptake are suboptimal.7 More than 30 percent of the U.S. population suffers from a mental illness each year.8 Of this group, 22 percent are classified as serious, 37 percent as moderate, and 40 percent as mild.9 In sum, about 17.8 million people (5.8 percent of the U.S. population) live with SMI in any given year, resulting in significant economic and societal consequences. Several working definitions of SMI, severe mental illness, or severe and persistent mental illness have been used.10-18 For this IEF's purposes, we used the following criteria to define adults with SMI:18 people who (1) are ages 18 or older; (2) currently have, or at any time during the past year had, a diagnosable mental, behavioral, or emotional disorder of sufficient duration to meet diagnostic criteria specified within the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV) or the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) equivalent (and subsequent revisions); and (3) have functional impairment that substantially interferes with or limits one or more major life activities. Major life activities (in part 3 of the definition) include basic daily living skills (e.g., eating, maintaining personal hygiene); instrumental living skills (e.g., managing money, negotiating transportation, taking medication as prescribed); and functioning in social, family, an vocational or educational contexts.19 American adults living with SMI die about 25 years earlier than other Americans, largely owing to treatable medical conditions.20 In fact, many people with SMI do not seek any health care.21 On average, those with SMI report being totally unable to carry out their normal daily activities for 88 days per year compared with 4.7 and 1.9 days, respectively, for those classified as having a moderate or mild mental illness.9 SMI is the second-leading cause of disability in the United States for ages 15 to 4422 and accounts for between 5,000 and 10,000 disability-adjusted life years lost worldwide per year per 1 million population. In 2002, SMI was estimated to cost more than $100 billion in health care expenditures alone.23 Loss of earnings as a result of SMI was estimated to be about $193 billion, and disability benefits cost an additional $24.3 billion, resulting in a total of more than $317 billion spent on SMI in 2002.23 SMI represents the largest diagnostic category for people receiving Supplemental Security Income (SSI) and Social Security Disability Insurance (SSDI) payments from the Federal Government. Table 1 provides data on the burden of disease associated with SMI and mood and psychotic disorders. Table 1 The burden of serious mental illness: Mood and psychotic disorders. [PLEASE SEE PDF FOR FIGURE] We limited the scope of the IEF to adults with SMI who have psychotic or mood disorders because these disorders account for the majority of SMI and because our time and resources were limited. We did not include alcohol- and substance-related disorders, developmental disorders, anxiety disorders, or personality disorders, unless they co-occurred with a psychotic or mood disorder. Figure 2 illustrates the focus of the IEF and the relationship among SMI, mood disorders, and psychotic disorders. Figure 2 The relationship between SMI, mood disorders, and psychotic disorders (this IEF focused on adults with SMI who have psychotic or mood disorders). Sources: National Comorbidity Survey. Epstein J, et al. [PLEASE SEE PDF FOR FIGURE] The overarching purpose of this Issues Exploration Forum was to contribute to the establishment of priorities to guide CER/PCOR activities in mental health by engaging a diverse group of stakeholders. We aimed to focus on an area of mental health with significant unmet need despite available interventions and an area in which conducting CER/PCOR is likely to have an impact on reducing variation and uncertainty in clinical practice and outcomes, reducing methodological and conceptual uncertainty, and reducing disease burden. Additional objectives were to identify knowledge gaps in the area of SMI and to generate and prioritize topics for future CER/PCOR, including topics for evidence synthesis (i.e., systematic reviews, comparative effectiveness reviews)."
Hide

Abstract

Objective: To set priorities for new systematic reviews on the management of primary angle closure (PAC), using clinical practice guidelines and a survey of clinicians from the Asia-Pacific region. Methods: We restated the American Academy of Ophthalmology's Preferred Practice Patterns (AAO PPPs) recommendations for management of PAC into 42 answerable clinical questions. We asked registrants at the Asia-Pacific Joint Glaucoma Congress 2010 in Taipei to rate the importance of having an answer to each question for good patient care, using a Likert-type scale from 0 (not important at all) to 10 (highly important). To assess understanding, we first asked two knowledge assessment questions about the survey objective, and allowed retries if the response was not correct. Respondents were randomized to one of two questionnaires, which had different orders of the same questions. The survey and instructions were in English. Results: 175 agreed to participate in the survey, 132 responded (75.4% response rate), and 96 completed the questionnaire (54.9% usable response rate). For those who responded but did not complete, half stopped after the two knowledge assessment questions. Of 96 who completed, only 29 (30.2%) correctly answered the knowledge assessment questions on the first attempt. The distribution of importance ratings pooled from all questions was skewed to high importance (mean = 6.92; median = 8). The order of questions affected the importance ratings. We considered clinical questions to be of priority if at least 75% of respondents gave ratings above 5. Conclusions: Understanding the survey purpose and questions affected our response rate, and question order affected question importance rating. We identified a list of clinical questions on the management of PAC for which research should be prioritized. Where evidence does not exist, primary research and systematic reviews should be initiated to address these questions.
Hide

Abstract

OBJECTIVE: To test the feasibility of a framework for prioritizing new comparative effectiveness research (CER) questions related to management of primary open-angle glaucoma (POAG) using practice guidelines and a survey of clinicians. DESIGN: Cross-sectional survey. PARTICIPANTS: Members of the American Glaucoma Society (AGS). METHODS: We restated as an answerable clinical question each recommendation in the 2005 American Academy of Ophthalmology Preferred Practice Patterns (PPPs) regarding the management of POAG. We asked members of the AGS to rank the importance of each clinical question, on a scale of 0 (not important at all) to 10 (very important), using a 2-round Delphi survey conducted online between April and September 2008. Respondents had the option of selecting "no judgment" or "research has already answered this question" to each question in lieu of the 0 to 10 rating. We used the ratings assigned by the Delphi respondents to determine the importance of each clinical question. MAIN OUTCOME MEASURES: Ranking of importance of each clinical question. RESULTS: We derived 45 clinical questions from the POAG PPPs. Of the 620 AGS members invited to participate in the survey, 169 completed the Round 1 survey; 105 of 169 also completed Round 2. We observed 4 response patterns to the individual questions. Nine clinical questions were ranked as the most important: 4 questions on medical intervention, 4 questions on filtering surgery, and 1 question on adjustment of therapy. CONCLUSIONS: Our theoretical model for priority setting for CER questions is a feasible and pragmatic approach that merits testing in other medical settings. Copyright © 2010 American Academy of Ophthalmology. Published by Elsevier Inc. All rights reserved.
Hide